2020.1

Product Notices

On this page:

Breaking Changes

BREAKING CHANGE for 2020.1.13 - 2020.1.14

createTransformation and updateTransformation API status codes

Upon an upgrade from a 2020.1 maintenance release version before 2020.1.14 to 2020.2, the API status code response for the createTransformation and updateTransformation APIs will now be a 200 instead of a 201 status code upon a successfully created or updated transformation.

What should I do?

Check any custom application which calls the createTransformation or updateTransformation API to see if it uses the statusCode response. If it does, make sure any check for a "201" response instead looks for a "200" response.

BREAKING CHANGE for 2020.1.12 - 2020.1.13

Changes to the Pronghorn.getMethods permission for Custom Roles

In 2020.1.13, the GET /methods/registry API is no longer included in the Pronghorn.getMethods permission for custom roles. Access to the GET /methods/registry API is now included under the Pronghorn.getMethodsRegistry permission instead, and the Pronghorn.getMethods permission solely includes the GET /methods API.

What should I do?

If any custom role contains the Pronghorn.getMethods permission and does not need access to the GET /methods API, please add the new Pronghorn.getMethodsRegistry permission to that role in Authorization Manager.

BREAKING CHANGE for 2020.1.11 - 2020.1.12

$where in Workflow Engine API Calls

Due to a vulnerability with Mongo, the $where key is no longer supported as part of the query object for several API calls. The $where key is specifically used in Mongo to support a JavaScript expression or function to be used as part of a query. Any APIs that were vulnerable to the usage of $where will now return with an error if $where is included in the query. The following APIs are affected by this, with the relevant parameter noted for each.

API Parameter
POST /workflow_engine/getJobFromTaskQuery task_query
POST /workflow_engine/queryJobs query
POST /workflow_engine/jobs/find query
POST /workflow_engine/queryTasksBrief query
POST /workflow_engine/getTask query


BREAKING CHANGE for 2020.1.2 - 2020.1.9 -> 2020.1.10

Adapter Local AAA (adapter-local_aaa)

The following are breaking changes for the 2020.1.10 maintenance release.

API Removal

The table below shows a list of APIs that have been removed from the Local AAA adapter. This list represents all APIs removed in the 2020.1.10 maintenance release. Our deprecation process typically prevents breaking changes between maintenance releases. This is an exception as these specific calls were used for a unique scenario and never exposed to our customer base. Thus, we do not expect anyone is using them; however, in case they were accidentally discovered, we are mentioning them here with a directive to remove them.

What should I do?

Review any custom apps and adapters that might reference any APIs in the list below and remove them.

API Removed Description Deprecation Release Actual Removal Release Replacement
addUser Exposed as a task to add a new user to Local AAA. None 2020.1.10 None
changePassword Exposed as a task to update the password of a user. None 2020.1.10 None
getGroups Exposed as a task to get all Local AAA Groups. None 2020.1.10 None


BREAKING CHANGE for 2019.3.0-2019.3.7 or 2020.1.2 -> 2020.1.3

The following are breaking changes for the 2020.1.3 release.

Migration of array.toString Tasks to objectToString

For workflows using the array.toString task that was migrated or saved in a released version of IAP between 2019.3.0 and 2019.3.7 or between 2020.1.0 and 2020.1.2, the array.toString task has now been migrated to objectToString. Between IAP versions 2019.3.0 - 2019.3.7 and 2020.1.0 - 2020.1.2, the array.toString task was changed to include two additional parameters and a slightly different output. Starting with IAP version 2019.3.8 and 2020.1.3, the functionality of array.toString was reverted to the functionality that was prior to IAP version 2019.3.0. In other words, regarding the array.toString task, there is no longer a breaking change from 2019.2 workflows to its 2019.3 or 2020.1 equivalents; however, in making this change, any workflows that intentionally used the new output of array.toString will now be broken as a result of this change. Accordingly, the next section outlines how to find and fix this scenario.

Manual Steps for Migrated Workflows

For workflows where the array.toString to objectToString migration is breaking, a script is now provided to find workflows that contain a specific task (this task can be used to find any task as the need arises). Follow the steps below to find all workflows that contain objectToString and manually update any that are breaking.

  1. Download the following script:

    Note: This script is also found on an installed version of 2020.1.3 in:

    /opt/pronghorn-core/node_modules/@itential/app-workflow_engine/migration_scripts/

  2. Place the script in the following location (on the migrated environment that includes the 2020.1.3 maintenance release):

    /opt/pronghorn-core/node_modules/@itential/app-workflow_engine/migration_scripts/

  3. Run the script to find all workflows that contain the objectToString task.

    cd /opt/pronghorn-core/node_modules/@itential/app-workflow_engine/migration_scripts/

node findWorkflowTasks.js objectToString

  4. If necessary, update any workflow tasks that are now breaking by replacing them with array.ToString.

  5. Update any tasks that reference the output of array.ToString that have not been updated already by setting the reference task dropdown to stringified.

  6. Update any template variables that reference the previous arrayString output of array.ToString to now reference stringified. Remember to check any and all references that can occur subsequent to the array.toString task.

BREAKING CHANGE for 2020.1.1 -> 2020.1.2

RabbitMQ Properties Moved from Service Config to Profile

Upon upgrade to the 2020.1.2 maintenance bundle, the RabbitMQ properties will be automatically moved from all service configs to the profile. Additionally, these settings will be changed in the database; therefore all systems that share this database connection will be affected also. Please note, if you have a disaster recovery (DR) system in your setup, the changes made to the database will take effect once the first system is updated and affect all systems using that database. This basically means that all systems will need to be upgraded at the same time.

Rollback Instructions

Since the RabbitMQ properties will be auto-migrated from the service config to the profile, if a rollback is necessary, those properties must be restored manually to each relevant service config prior to upgrade.

BREAKING CHANGE for 2019.3.1/2019.3.2 -> 2020.1

The following are breaking changes for the 2020.1 release, specifically when updating from the first two maintenance releases of 2019.3 directly to 2020.1.

Important Note

It is the policy of Itential not to cause any breaking changes within the maintenance releases of a major release version; however, at times we have to do so and we extend our sincere apologies when those times occur. This situation occurred in maintenance release 2019.3.3.

Please note, Itential removed two features:

  1. Translations, and
  2. Multiple job variables on the "Outgoing" tab of automatic tasks.

The following procedure is only necessary if upgrading from 2019.3.1 or 2019.3.2 directly to release version 2020.1. If you have already performed these steps in an update from 2019.3.1/2019.3.2 -> 2019.3.3, then they are not needed when upgrading to 2020.1. There is no harm, however, in double-checking to be sure whether or not these steps may apply.

Removing the Translations Feature

For the 2019.3.0 IAP release, Itential deprecated all the global object methods (string, number, etc.) in Workflow Builder and replaced them with a 'translations' feature. Once that deprecation was made, Itential heard from many customers that the manual effort to replace all those methods would be a very large one with the toolset that is provided. With that level of effort in mind, and our commitment to provide customers with the best user experience possible, Itential decided to un-deprecate those methods in the 2019.3.3 maintenance patch.

Please note, the translations feature was removed, which means you cannot use translations in your new workflows on development servers. The global object methods will be available going forward beginning with IAP 2019.3.3.

Manual Steps to Ensure Database Integrity

Automations (within the Automation Studio) no longer allow tasks that export more than one job variable or that attach translations to exported job variables within automations. Workflow Engine will not support running automations with tasks that have translations on your outgoing data, or export more than one job variable as outgoing data. In the 2020.1 release, Workflow Engine will not start if it finds an automation that meets these conditions.

These automations need to be corrected manually. Use the following process:

  1. Download the following two scripts:

    Note: These scripts are also found on an installed version of 2020.1 in:

    /opt/pronghorn-core/node_modules/@itential/app-workflow_engine/migration_scripts/

  2. Place the script files in the following location of a 2019.3.1 or 2019.3.2 system:

    /opt/pronghorn-core/node_modules/@itential/app-workflow_engine/migration_scripts/

  3. Run the script to find automations that have incompatible tasks.

    # cd /opt/pronghorn-core/node_modules/@itential/app-workflow_engine/migration_scripts/

# node compatibilityCheck20193Maintenance.js

  4. The output of the script will indicate which automations need to be corrected.

    IMPORTANT: The correction must occur on a system running 2019.3.1 or 2019.3.2.

  5. After all automations have been updated, run the script again to ensure they are compliant.

  6. Once compliant, proceed with the update to 2020.1.

Sample Output

# cd /opt/pronghorn-core/node_modules/@itential/app-workflow_engine/migration_scripts/`

# node compatibilityCheck20193Maintenance.js

incompatibleWorkflow contains incompatible tasks:

    getAdaptersForDevice (508a)
    Task Summary: exports more than one job variable
    Reason: Task exports more than one job variable, each of which may or may not have translations attached to it.


    locateDevices (e7c)
    Task Summary: exports a job variable with a translation
    Reason: Task exports a job variable with a translation attached to it.


    locateDevices (37ec)
    Task Summary: exports more than one  job variable with translations
    Reason: Task exports more than one job variable, each of which may or may not have translations attached to it.


    RunCommand (2aed)
    Task Summary: exports more than one job var with no translations
    Reason: Task exports more than one job variable, each of which may or may not have translations attached to it.

Note: For ease of reference, the script output provides the name of the automation (e.g. incompatibleWorkflow) and the taskname (e.g. getAdaptersForDevice and locateDevices),

This compatibility check script inspects all automations in your IAP deployment and will deem an automation incompatible if it contains any tasks that:

  • Export more than one job variable (each of which may or may not have a translation).

  • Export a job variable with a translation.

Modify the Automation to Fix Incompatibility

To fix the first case, modify the automation so that tasks exporting more than one job variable now:

  • Only export a single job variable without any translations.

  • Use the newVariable task to instantiate the number of job variables that the incompatible task created.

  • Use the appropriate tasks from Number, Object, String, and Array to perform data manipulation on created job variables (if necessary).

Modify the Task to Fix Incompatibility

To fix the second case, the task on an incompatible automation should be modified such that:

  • It still exports a single job variable via its outgoing data.

  • All translations are removed.

  • The appropriate tasks from Number, Object, String, and Array are used to modify the exported job variable the same way that the translation did.

As stated above, once all workflows have been cleared of any tasks that either have multiple job variables defined or have a translation, the update to 2019.3.3 is cleared to proceed.

BREAKING CHANGE for 2019.3.2 and below -> 2020.1

Job Manager Database Enhancement

The following procedure is only necessary if upgrading from 2019.3.2 or lower directly to release version 2020.1. If you have already performed these steps in an update to 2019.3.3 or higher, then this procedure is not needed when upgrading to 2020.1. There is no harm, however, in double-checking to be sure whether or not these steps may apply.

As part of an ongoing effort to always provide the best performance within IAP, this release contains a migration script which will optimize the jobs data to allow faster queries against the collection. There are two possible methods to run this migration:

  • Ensure you have a full Mongo backup of your database before following this procedure.

  • npm run normalizeGroups from the app-workflow_engine directory. The script will modify each job within the database, so depending on the number of jobs it may take 5 - 15 mins to complete. This is the slowest method because it is run from the IAP server and is only recommended for instances where the jobs collection is moderately sized (low 10Ks number of records).

  • Download the normalizeGroups.js script and run it on the Mongo server via the command line. For example: mongo mongodb://HOSTS:PORT/DATABASE?replicaSet=rs_pronghorn --authenticationDatabase admin -u USER -p PASS < normalizeGroups.js. Tailor the MongoDB URI to your environment. This method is much faster and is recommended for large jobs collections.

These changes should not have any impact on your usage of Job/Task history.

Note: With this breaking change, it is very important that you ensure all indexes are setup correctly on Jobs and Tasks. This can be done by running the npm run index command from within app-workflow_engine. If you are missing indexes, it may take 15-30 mins to complete the index creation on large databases.

BREAKING CHANGES for 2019.3.* -> 2020.1

Policy Manager (app-policy_manager)

With the addition of network groups and service groups, some changes were made to the structure of Policy Rule objects. Specifically, references to networks and services, previously ids (e.g. "abc123"), are replaced with typed references (e.g. { "type": "networkGroup", "id": "abc123" }). These changes affect payloads and results of the following methods.

Methods
getPolicies / getRuleTemplates
createPolicy / createRuleTemplate
addRuleToPolicy / addRuletoTemplate
modifyPolicyRule / modifyTemplateRule
updatePolicyRules / updateTemplateRules


Automation Engine (app-workflow_engine)

IAP 2020.1.0 includes the release of an improved Automation Engine. Incomplete tasks are no longer inserted into the tasks collection. As a result, the returned payload of the following methods are affected (e.g. incomplete tasks are not returned with these calls).

Methods
searchTasks
queryTasksBrief
getTask
queryTasks
getTaskIterations
getJobFromTaskQuery
getAssociatedJobs


The inputs for getManualTaskController have changed to include both the task id and the job id, instead of just the id of the task document.

To account for this change, please update any custom code and any automations using these APIs.

For getAssociatedJobs, the response now includes the task data available within a job document with a modified owner property (that is the same as previous releases). The biggest change is that tasks returned using this call no longer have _id, but instead have taskId and an array of iteration ids that are similar to the previous _id. See the comparison below.

2019.3 Data

"owner": {
    "_id": "5e1ca0b7ede734677b0a37b7",
    "provenance": "Local AAA",
    "username": "admin@pronghorn",
    "firstname": "admin",
    "memberOf": [],
    "assignedRoles": []
    "_meta": {},
    "email": ""
},
"_id": "53504ab0-ecc2-469a-91d4-bf12d702c5c3",

2020.1 Data

"taskId": "25ec",
"owner": {
    "_id": "5e1ca0b7ede734677b0a37b7",
    "provenance": "Local AAA",
    "username": "admin@pronghorn",
    "firstname": "admin",
    "memberOf": [],
    "assignedRoles": []
    "_meta": {},
    "email": ""
},
"iterations": [
    "53504ab0-ecc2-469a-91d4-bf12d702c5c3"
],

Task Worker (app-task_worker) Properties Moved to Automation Engine

Task Worker is now part of app-workflow_engine. As a result, the activate and TICK properties (found within the Task Worker Service Properties UI) are no longer used by any app. Since the TICK property will no longer be used, there will be no effect on the system. If the activate property has been set in your current environment, please ensure it is now set within the WorkFlowEngine Service Properties UI. This replaces the app-task_worker activate property and operates in the same way.

Itential Tools (itential_tools)

With each new release, NSO capabilities are becoming more enhanced and providing more features out of the box. Hence, some of the Itential Tools methods became redundant and obsolete. Below are the methods deprecated in 2019.2 and removed in the 2020.1 release.

Method Replacement
createTemplate NSO provides create_template out of the box.
doCommand Use either runCommand or runAction.
native2XML No replacement.
getYANG No replacement.


Adapter NSO (adapter-nso)

Below are the adapter-nso methods deprecated in 2019.2 and removed in the 2020.1 release.

Method Replacement
getConfig Use "CISCO_AR_FORMAT" instead of "NATIVE" as the format input parameter.
getSchema getServiceModel
getYANG getServiceModel
loadXML NSO Manager:runAction
passThru runCommand
loadNativeConfig NSO Manager:runAction
createGroup saveGroup
getReports No replacement.
createReport No replacement.
deleteReport No replacement.
runCompliance No replacement.
getServiceDeviceList getDevicesFiltered
listServiceModels getServiceModelMap
getDeviceModificationsForServiceInstance getDeviceModificationsForServiceInstanceWithParams
deleteService deleteServiceWithOptions
deleteServicePoint No replacement.
prepareServiceData No replacement.
addServiceInstance saveServiceInstances
addServiceInstanceWithOptions saveServiceInstances
addServiceInstanceFromObject saveServiceInstances
DryRunService testServiceInstances
dryRunServiceInstance testServiceInstances
dryRunServiceInstanceObject testServiceInstances
getServiceInstanceObjectTemplate getServiceModel
testServiceInstance testServiceInstances
saveServiceInstanceObject saveServiceInstances
saveServiceInstance saveServiceInstances
convertObjectToArray No replacement.
getServiceModelFields getServiceModel


Smart Template (app-smart_template)

The following app-smart_template method was deprecated in 2019.2 and removed in the 2020.1 release.

Method Replacement
task/TemplateForm task/SmartTemplateForm